Papers with Recurrent Neural Networks
LTRC-MT Simple & Effective Hindi-English Neural Machine Translation Systems at WAT 2019 (D19-52)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is a promising approach for low resource languages. |
| Approach: | They propose to use both Recurrent Neural Networks & Transformer architectures to train NMT models. |
| Outcome: | The proposed model outperforms Statistical Machine Translation (SMT) techniques on a low resource Hindi-English language pair. |
Exploring Statistical and Neural Models for Noun Ellipsis Detection and Resolution in English (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to noun ellipsis resolution have been sparse, using syntactic feature constraints for marking licensors and selecting their antecedents. |
| Approach: | They propose to use supervised machine learning to improve the existing F1 score by 16.55% and resolution by 14.97% for noun ellipsis subtasks. |
| Outcome: | The proposed framework improves the existing F1 score by 16.55% and the resolution subtask by 14.97%. |
Multilingual, Multi-scale and Multi-layer Visualization of Intermediate Representations (D19-3)
Copied to clipboard
| Challenge: | Currently, the main alternatives to deal with sequences are Recurrent Neural Networks (RNN) architectures and the Transformer. |
| Approach: | They propose a web-based tool that visualizes the sentence and token representations of RNNs and Transformer architectures at the sentence level. |
| Outcome: | The proposed visualization tool analyses gender inequalities in contextual word embeddings and the common language representation in a multilingual machine translation system. |
Transition-based Parsing with Stack-Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing parsing systems use local or global models of the parser state to improve performance. |
| Approach: | They propose to modify the sequence-to-sequence Transformer to model global or local parser states in transition-based parsing. |
| Outcome: | The proposed model significantly improves performance on dependency and Abstract Meaning Representation (AMR) parsing tasks. |
On the Practical Computational Power of Finite Precision RNNs for Language Recognition (P18-2)
Copied to clipboard
| Challenge: | Recurrent Neural Networks (RNNs) are famously known to be Turing complete, but this relies on infinite precision in the states and unbounded computation time. |
| Approach: | They propose to use LSTM and Elman-RNN with ReLU activation to study RNNs . they show that LS and ReLU-RNns can easily implement counting behavior . |
| Outcome: | The LSTM and the Elman-RNN with ReLU activation are stronger than the RNN with squashing activation and the GRU. |
Efficient Convolutional Neural Networks for Diacritic Restoration (D19-1)
Copied to clipboard
| Challenge: | Diacritic restoration is a computational task that requires a computer to understand written texts. |
| Approach: | They propose to use Temporal Convolutional Neural Networks (TCN) to restore missing diacritics for each character in written text. |
| Outcome: | The proposed model improves on TCN in Arabic, Yoruba, and Vietnamese. |
Enhancing Modern Supervised Word Sense Disambiguation Models by Semantic Lexical Resources (L18-1)
Copied to clipboard
| Challenge: | Existing supervised models for Word Sense Disambiguation (WSD) are limited to knowledge-based approaches. |
| Approach: | They propose to use WordNet and WordNet Domains to enhance supervised WSD models by introducing semantic features into the classifiers and using the SLR structure to augment training data. |
| Outcome: | The proposed model improves the state-of-the-art in Word Sense Disambiguation (WSD) The proposed approach is compared with the state of the art in the most popular benchmarks. |
PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism (2023.acl-long)
Copied to clipboard
| Challenge: | Existing generative models for dialogue use the last hidden state to summarize the history of the dialogue. |
| Approach: | They propose a Pseudo-Variational Gated Recurrent Unit (PVGRU) that summarises the accumulated distribution variations of subsequences and builds a model based on it. |
| Outcome: | The proposed model can improve diversity and relevance of responses on two benchmark datasets. |
To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness (P18-1)
Copied to clipboard
| Challenge: | Recent success of Recurrent Neural Networks (RNNs) in Machine Translation (MT) has prompted attention mechanisms to be used in machine translation. |
| Approach: | They propose a tree-structured attention model on Tree Long Short-Term Memory Networks . they also experiment with three LSTM variants: bidirectional-LSTMs, Constituency Tree-LSTS, and Dependency Tree LSTS. |
| Outcome: | The proposed model is based on tree-LSTMs, constituency trees, and dependencies trees. |
GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling (P19-1)
Copied to clipboard
| Challenge: | Existing systems for sequence labeling are limited by shallow connections between consecutive hidden states and insufficient modeling of global information. |
| Approach: | They propose a global context enhanced deep transition architecture for sequence labeling . they deepen the state transition path at each position in a sentence and assign tokens with global representations . |
| Outcome: | The proposed architecture outperforms the best reported results on two standard sequence labeling tasks. |
On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing studies on crosslingual transfer have focused on word-level information sharing, but words are not independent in sentences; their combinations form larger linguistic units, known as context. |
| Approach: | They propose to use orderagnostic models to transfer word order to distant languages . they train dependency parsers on an English corpus and evaluate their transfer performance on 30 other languages. |
| Outcome: | The proposed model performs better on languages with different word orders than on other languages. |
Neural Caption Generation for News Images (L18-1)
Copied to clipboard
| Challenge: | Existing methods for automatic caption generation of images are lacking in the field of image-related applications. |
| Approach: | They propose a method for automatically generating captions for news images . they propose several deep neural network architectures built upon Recurrent Neural Networks . |
| Outcome: | The proposed method outperforms a traditional method on a BBC News dataset using automatic evaluation and human evaluation. |
Attention-based Conditioning Methods for External Knowledge Integration (P19-1)
Copied to clipboard
| Challenge: | Existing approaches for incorporating external knowledge into deep neural networks (RNNs) lexicon features are used to concatenate external information into the input or hidden network layers. |
| Approach: | They propose a method for conditioning external knowledge into RNNs by concatenating a representation of the external information to the input or hidden network layers. |
| Outcome: | The proposed approach improves performance on six benchmark datasets. |
Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities (2020.coling-main)
Copied to clipboard
| Challenge: | Domain names such as openresearch are being added to a growing set of tokens that an NLP system may need to deal with. |
| Approach: | They propose a tagging model that uses characters as input to break domain names into component words . they propose taagging methods that use concatenated entity names in a large knowledge database . |
| Outcome: | The proposed model improves on concatenated entity names in a knowledge database by 33% . the proposed model can be used for a wide range of languages, including Chinese and Japanese . |
Word-Level Uncertainty Estimation for Black-Box Text Classifiers using RNNs (2020.coling-main)
Copied to clipboard
| Challenge: | Neural Networks are not interpretable, since they provide no information about why particular decisions were made. |
| Approach: | They propose to decompose and visualize uncertainty of text classifiers at the level of words to provide detailed explanations of uncertainties. |
| Outcome: | The proposed approach decomposes and visualizes uncertainty of text classifiers at the level of words and enables a deeper understanding of unreliable model behaviours. |
DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing retrieval-based dialogue systems suffer from slow inference or huge number of parameters. |
| Approach: | They propose a lightweight fully convolutional architecture for response selection using convolution. |
| Outcome: | The proposed architecture extracts matching features of context and response from 3D views. |
Seq2SeqPy: A Lightweight and Customizable Toolkit for Neural Sequence-to-Sequence Modeling (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural models have attracted a lot of attention in the past few years due to their complexity and need to be customized to meet specific needs. |
| Approach: | They propose a lightweight toolkit for sequence-to-sequence modeling that prioritizes simplicity and ability to customize the standard architectures easily. |
| Outcome: | The proposed tool performs similarly or even better than a very widely used sequence-to-sequence toolkit. |